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Abstract 


This study is based on the creation of a theoretical tool for determining 
competencies and knowledge in science education which stakeholders value 
as necessary for students to become productive members of the future work 
force as well as responsible citizens. Stakeholders ’ opinions (N=85) were 
gathered from persons in different walks of life by means of a Delphi study, 
conducted in two rounds, while a second part of this study set out to 
evaluate whether the Delphi method is effective in achieving the cam of the 
study. Results from the first round indicate that stakeholders expect future 
employees to possess higher levels of scientific literacy, although this is far 
from being unanimous. This is confirmed in the second round where views 
held by stakeholders were found to be in line with those expressed in the 
science literature. The article indicates that Delphi study approaches have 
merits, but steps are required to avoid typical difficulties such as contacting 
participants and then keeping theme motivated throughout the research 
process and in overcoming obstacles which arise. Results from this study 
are important for both curricular and national development where the goal 
is to move towards a more knowledge-based society. 

Keywords: Delphi study, science education, scientific literacy, stakeholders 

Introduction 

The present situation in science education reflects a concern: the European Commission 
report (2004) states that students’ attitude towards science is poor while outcomes from the 
PISA2006 study showed science was not an interesting subject in many countries (OECD, 
2007). This relates to previous research which has shown that students are not interested in 
science as well as not wanting to associate their future career with science (Osborne et al., 
2003). This supports the Estonian Research, Design and Innovation strategy document 2007- 
2013 “Knowledge-based Estonia” which states that the biggest weaknesses in becoming a 
competitive country in Europe and the world are the small number of graduates in natural 
sciences and technology (this may be caused by the negative attitudes towards science) plus 
the lack of attention paid to research and innovation. Yet little is known whether this situation 
derives from attitudes of the society, students’ interests in their future aspirations, or the type 
of curriculum being promoted in schools. Of particular interest in this respect is the range of 
stakeholder views on what educators, employers and the like are wishing students to gain 
from studies in science education. 
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Scientific literacy 

The current situation in Estonia is well highlighted (OECD, 2007; Henno et al., 2008): 
students’ theoretical knowledge in science is above the OECD average, but there is a general 
lack of inquiry skills and the ability to use knowledge which directly impacts on students’ 
scientific literacy. These findings are quite similar to those found in general by Bybee and 
Fuchs (2006) and in line with scientific literacy achievements (Laugksch, 2000). To achieve 
greater economic competitiveness between countries indicates the need for scientific 
excellence and technological innovation, as well as a strong research and development base 
(EU, 2004); students need to be provided with the appropriate skills and knowledge for 
achieving higher levels of scientific and technological literacy. 

Many researchers have stated that scientific literacy should be the goal of science education 
(e.g. Laugksch, 2000; Fensham, 2004; Holbrook & Rannikmae, 2007) and there is a need for 
a scientifically literate workforce and citizens (DeBoer, 2000; Collins et al., 2001; Fensham, 
2004; Brown, Reveles & Kelly, 2005; Bybee & Fuchs, 2006). Gauld (2005) even went as far 
as to suggest science has an impact on everyone in some way and many scientific issues 
become political in nature. Whatever the reason, it seems that while future societies need 
people who are able to use scientific knowledge to solve problems in everyday-life or work 
situations, schooling is not providing society with sufficient people who have such abilities. 

The gaining of scientific literacy can be expressed by using hierarchical levels (Bybee, 1997). 
This recognizes that scientific literacy is not simply factual knowledge and by use of the 
levels it is possible to demonstrate a persons’ ability to utilize scientific knowledge when 
solving upcoming problems. Bybee suggested that the first level, which he called nominal 
scientific literacy, is illustrated by a person who recognizes when a term, question or topic is 
scientific in its nature, but, however, demonstrates clear misunderstanding. An individual’s 
understanding is minimal in comparison to the accepted scientific understanding for the 
individual’s age and situation. 

Bybee termed the second level ‘functional scientific literacy.’ At this level a person is able to 
use scientific and technological vocabulary in a particular activity when needed (e.g. defining 
terms in a test, reading a newspaper, or listening to a television program), but it is generally 
out of context and lacks the conceptual embellishment of disciplines. Unfortunately, 
functional scientific literacy has been the ultimate goal of science education for a long time 
and much emphasis has been placed on attaining this level. This finding is supported by 
Osborne and Dillon (2008) who claim that science teachers’ pedagogy is still dominated by a 
focus on content rather than developing the understanding of science. 

The third level, conceptual and procedural scientific literacy, describes persons who have 
developed an understanding of the way conceptual parts of a discipline relate to the whole and 
how scientific disciplines relate to each other. They possess procedural knowledge and skills 
(e.g. scientific inquiry skills, technological skills, ability to make observations and hypothesis, 
developing new knowledge using evidence, logic and creativity). In Shen’s (1975) words, it 
can be described as the level of scientific ability which enables the solving of practical 
problems. 

The highest level, multi-dimensional scientific literacy, illustrates persons who have an 
understanding of science which extends beyond the concepts of scientific disciplines and 
procedures of scientific investigation. More specifically, such persons are able to make 
connections within scientific disciplines, and between science, technology, and the larger 
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issues challenging society. In other words, science education has promoted a broader view of 
science, while simultaneously helping to foster an appreciation for science and its usefulness 
to society (Roberts, 2007). 

Clearly, as Bybee suggests, the multi-dimensional level is seen as a key target for science 
education. Such scientific literacy, gained through science education, enables the creative use 
of sound scientific knowledge in everyday life or in a career, to solve problems, make 
decisions and hence improve the quality of life (Holbrook & Rannikmae, 2009). In this view, 
scientific literacy is important for both a personal and professional life. It also highlights that 
besides the capability for problem solving, enhancing scientific literacy also enables a person 
to improve one’s life. In other words, it is associated with the capability to transfer 
knowledge, skills, attitudes and values to unknown situations such as showing initiative, 
thinking critically or reasoning oneself in a collaborative working situation. 


In order to promote multi-dimensional literacy, it is important to determine from where the 
impetus to promote such a development is more likely to come. Teachers perceive themselves 
as confined by the curriculum and curriculum goals and students and society in general seem 
to be accepting the current school situation. One option is to seek views from a range of 
stakeholders, or experts without references to school and school science teaching. This can be 
undertaken by focusing on careers after school and what is perceived as important from that 
perspective. 

Knowledge and Competencies 

Both the terms knowledge and competency have multiple meanings and are difficult to 
express with precision in everyday language. This makes it difficult to seek opinions from 
stakeholders and then interpret them in a meaningful way. In this article, knowledge is taken 
to refer to a range of cognitive attributes from factual recall of information to higher level 
analytical or evaluative abilities. On the other hand, competencies are taken to represent a set 
of mental capabilities, or skills, attitudes and values which students need to acquire to 
progress towards the capability to meet curriculum competence standards, set within a 
particular domain (Gilmer, Sherdan, Oosterhof, Rohani & Rouby, 2011). Fensham (2007) 
goes further and suggests that competencies are abilities to put elements from one knowledge 
system together with elements from another, arranging them so that they work in new ways 
and do new things. In other words, competencies gained in science classes involve, for 
example, reasoning with evidence and argumentation (Brickhouse, 2007), while 
competencies, taken from a more general educational view, are the capabilities, expressed as 
learning outcomes, that will be increasingly needed (Fensham, 2007). It is clear that messages 
from employers and policy makers, indicating employment competencies that young people 
should possess, regardless of their relative success in formal academic education, are 
important for today’s personal, social and economic life (OECD, 2000). 

Competencies play an important focus in the new Estonian curriculum for gymnasium level 
(2011) where it is describes as follows: competency is a combination of pertinent knowledge, 
skills and attitudes which enable to be successful in a particular field of activity. Science 
education in the new curriculum indicates the need to promote competencies such as 
understanding relationships between humans-nature-environment, being able to pose research 
questions and participating in conversations about different scientific fields. While these 
competencies, also emphasized in reports by the EC (2006) and Eurydice (2009), are seen as 
important, it is clearly desirable that the value of these competencies are recognized by 


204 



Stakeholder views on attributes of scientific literacy important for future 

citizens and employees - a Delphi study 


society and especially by employers and other stakeholders. This suggests that the need for 
promoting science related careers and also the scientific literacy level of future members of 
society requires consideration of stakeholder’s views and strong compatibility between these 
views and the competencies being promoted in the science curriculum. 

The Delphi method involving stakeholders 

A stakeholder or expert is someone who will be directly affected, or has an applicable 
specialty, or relevant experience (Scheele, 2002). Stakeholders or experts in this particular 
context are those who hold wide competency-based views on science education goals or have 
a direct interest in the outcomes of Estonian science education. A major reason to consider 
stakeholders’ views is that whatever change is contemplated, it is more likely to happen when 
people whose thoughts ‘matter more’ are involved and actions take the desired course. 
Stakeholders or experts can also be those who provide employment to future citizens and thus 
know the directional trends in which the society and its needs are moving, or the kind of 
competencies or knowledge which is valued in a competency-based society where scientific 
excellence and technological innovation is the key to success. A consolidation of participants’ 
opinions enables identification of the diversity of expectations from stakeholders, which can 
then be compared with the familiar goals of science education. Based on this, it is important 
to disseminate such views to those who directly decide how science should be taught in 
science classes. Informed stakeholders in a particular field related to student needs and using 
their informed knowledge and a cultivated sense of relevance are best able to carry out 
reasoned predictions in his field (Pollard & Pollard, 2004). 

One method that is widely used to collect the views and obtain a consensus from a range of 
stakeholders or experts is the Delphi method (also Delphi study, Delphi technique or simply 
Delphi) (Pollard & Pollard, 2004; Skulmoski, Hartman & Krahn, 2007). The method itself 
was introduced in 1950 with the aim to determine national security risks, but later gained 
popularity in other fields, including research in education (Chypert & Grant, 1970; Listone & 
Turoff, 2002; Pollard & Pollard, 2004). The Delphi method is an iterative process used to 
collect and distil the judgments, views or thoughts of experts using a series of questionnaires 
or questions, interspersed with feedback. The method is usually undertaken in several rounds 
or stages, where the number of rounds varies. In most cases it is considered necessary to 
introduce at least two rounds (Pollard & Pollard, 2004; Collins et al., 2001; Skulmoski et al., 
2007), where each subsequent instrument is developed based on the results of the previous 
one. The process stops when consensus is reached, or when sufficient information has been 
exchanged (Listone & Turoff, 2002). 

The main reason for using a Delphi study in this study is to collect information or views from 
a wide range of stakeholders or experts and at the same time make stakeholders aware of 
potentially useful views coming from others. The aspect of interest is then to determine the 
degree of consensus between the stakeholders involved. Although this method has its 
advantages, there are also disadvantages or weaknesses which can inhibit the effectiveness of 
the process. The following table (Table 1) illustrates the main disadvantages of using the 
Delphi method for this study and the precautionary steps taken to avoid or minimize these 
disadvantages. 

The aim of this study is to seek different stockholder’s opinions about the competencies and 
knowledge to be gained from science education, which a school leaver should have in order to 
be successful in the workforce and/or as a citizen in society. The methodology for seeking 
stakeholder’s views about science education is described in terms of levels of scientific 
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literacy (Bybee, 1997). Based on this, a scale of scientific literacy levels is created which is 
intended to reflect stakeholders’ opinions. 


Table 1 . Steps taken to avoid the main limitations of using the Delphi method 


The limitations of using the Delphi method* 


Precautionary steps to minimize limitations 


The process is time-consuming and thus participants 
need to have sufficient time for participation. 

The selection process of experts or stakeholders 
(experts should be able to answer the research 
questions) is a critical part because the participants’ 
opinions are crucial. 

Researchers influence on the response due to a 
particular question formulation. 


The participants were informed of the nature of the 
study and asked whether they have enough time before 
the study was launched. 

The selection process was carried out with others from 
the field of science education. A total of 4 persons 
were included in the selection of participants. 

The research question was piloted and validated by 
others in the field of science education. 


*Pollard & Pollard, 2004; Murray & Hammons, 1995; Skulmoski et ah, 2007; Listone & Turoff, 2002. 


The study is based on the following research questions: 


1. What competencies and knowledge do stakeholders hold with the respect to the 
development of science education and how can these views be interpreted in the light 
of the need for students to attain scientific literacy? 

2. How appropriate is the Delphi study approach in meeting the aim of this study? 


Methodology of Research 


Study design and sample 

In this study, two rounds were used to collect data as illustrated in figure 1. 



Figure 1 . The design of the study. 


The aim of the round one open-ended question (“What kind of knowledge and competencies 
in science education do you expect an individual to possess when he or she enters the labor 
market and/or society?”) was to find out, with the respect to science education, participants 
expectations for individuals to possess when they enter the labor market and/or society. This 
open ended question was first piloted with a small number of individuals randomly chosen, 
with the aim of finding out the main obstacles in using a Delphi study as a method and to gain 
an indicator of types of problems and answers which were likely to be given. The feedback 
obtained, as well as a carefully conducted validation exercise using 4 experts, was used to 
modify the question for the first round to that quoted above. 
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The selection of stakeholders was based on the fact that all had expertise in the process and 
practices related to their field of activity which could be connected to the outcomes of science 
education. The selection process ensured that both the private and public sectors were covered 
as well as scientists from the natural and social sciences. Participants selected for the study 
(approximately equal numbers per category, see table 2), were first informed of the nature of 
the study and asked whether they were willing to participate. In total, 38 stakeholders agreed 
to participate. 


Table 2. Number of participants involved in the study given in 6 categories. 



Public 

sector 

employers 

Private 

sector 

employers 

Scientists 

Science 

teachers 

Students 

studying 

aboard 

12 th grade 
students 

Total (N) 

Round 1 

8 

6 

8 

7 

3 

6 

38 

Round 2 

8 

6 

5 

6 

3 

57 

85 


The teachers selected for the study were all science teachers teaching in secondary level and 
thus represented the current direction of science education in school. Insofar as was possible, 
Estonian students studying in colleges around Europe were also included as they were 
perceived as having experience within another system and could be able to give opinions 
about science lessons from another perspective. Students from grade 12 were also included in 
the study because, after graduating, their knowledge and skills would be needed in society. 
For the first round, all the 12 th grade students were studying in the same class. 

After piloting, the first round of the study was carried out as follows: all participants received 
an e-mail including the round one question and were asked to respond to the e-mail by writing 
their own opinons in response to that question. None of the participants had previous 
experience with a Delphi study. 

After the first round was completed, the responses were analyzed, categorized, checked by 
two experts in the field and then reformulated as statements in a 5 point Likert scale 
questionnaire. These statements (44 in total) were then also categorized by experts (2 experts 
from the field of science education knowledgeable in the scientific literacy literature working 
independently and then agreeing to categories by consensus) into the 4 levels of scientific 
literacy suggested by Bybee (1997). 

The 44 statements, in a 5 point Likert scale questionnaire (ranging from 1 being not important 
at all to 5 being very important), were sent to the participants as round two of the Delphi 
study so as to determine their opinions about the competencies and knowledge that had been 
identified in round 1 from all responses. The stakeholders were asked to indicate the 
importance of each statement with respect to science teaching. The stakeholders were the 
same as in round one, although 4 participants (1 teacher and 3 scientists) withdrew. Although 
this might have affected the results, it was recognized that the minimum number of 
participants to obtain meaningful data is considered to be ten (Pollard & Pollard, 2004). 
However, in order to get wider, more comprehensive views from school leavers, the number 
of 12 th grade students involved was increased by including an additional 51 students (total 
12 th grade students = 57). This was considered important because it enabled a comparison of 
views gained from stakeholders with the views of students soon to enter the workforce. 
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Results and findings 

Results and findings from round one 

These results are based on the replies from the 38 participants answering the initial central 
question “What kind of knowledge and competencies in science education do you expect 
individuals to possess when they enter the labor market and/or society?” The results showed 
that there is a strong consensus among different stakeholders about what kind of knowledge 
and competencies was essential, although stakeholders emphasized the importance of 
competencies in different fields or subjects, especially in science. But at the same time, it was 
noticeable that an emphasis was placed on gaining knowledge rather than the use of 
knowledge. In this respect, the results clearly illustrated the current situation in school, where 
studies have shown that students possess good knowledge, but they do not know how to use it 
(Henno et al. 2008). 

Without an awareness of the term scientific literacy, many participants emphasized 
competencies used when talking about a scientifically literate person. Yet from an analysis of 
the 12 th grade student’s answers, it became clear that they only emphasized knowledge gained 
from school and did not see the need for knowledge and skills gained outside of school. 
Students’ answers included more of the elements of school science topics, although they still 
emphasized practical skills considered important in their future life. Science teachers, as well 
as students, emphasized more than other groups the importance of factual knowledge gained 
in science classes. Also, compared to participants from the private and public sector, who 
emphasized the importance of practical knowledge, teachers still thought it was more 
important to know laws and facts instead of the ability to use knowledge. 

Based on the responses from the first round, individual statements, covering the separate 
points made, were compiled into groups according to similarities of themes. The essential 
competencies, skills and knowledge from these groups were then put into one statement to 
comprise the 44 statements. 

Results and findings of round two 

An analysis of the 44 statements showed that none could be assigned to the first level of 
scientific literacy (see table 3) and hence the first level was not considered further. 

Results were obtained from 85 respondents and analyzed as shown in table 4. Responses from 
the 44 statements were included for each group where (a) an overall mean value was 4.00 or 
higher and (b) the mean value was 3.00 or lower. The main reason for this selection was 
because stakeholders had already given their opinions on the original research question and 
thus these statements represented areas of relative importance, or non-importance for them. 
This group selection process omitted those statements for which the emphasis was not so 
apparent. All statements included in the questionnaire are given in appendix 1 together with 
the identified science literacy category level. 

The number of statements, with mean values 4.00 or above, or 3.00 or below, for the six 
groups is indicated in table 4. The actual statements are given in appendix 1. 

Table 4. The division of statements between the levels of scientific literacy among 6 study 
groups, given as number of statements and a percentage of the total statements at that level 
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Table 3. The division of statements between the levels of Scientific Literacy (SL). 


Levels of SL 

Description of the level 

Number /(%) of 
statements 
(N=44) 

Examples of statements 

2 nd level- 
functional 

Is able to use scientific vocabulary 
when needed; memorize and 
reproduce scientific knowledge; 
knowing what has been taught in 
school 

17(36%) 

ability to memorize and 
reproduce facts and 

formulas taught in school 
(statement 16) 

3 rd level- 
conceptual and 
procedural 

Is able to use knowledge and skills 
from science and technology, logic, 
observational and inquiry skills 
when solving problems in work and 
home and cooperate with other 
people 

20(48%) 

ability to recognize science 
content in everyday life 
issues (statement 41); 
ability to examine/observe 
processes or phenomena in 
the nature (statement 2) 

4 th level¬ 
multidimensional 

Is able to use scientific knowledge in 
problem solving; is able to connect 
different areas of science into one 
system when dealing with societal 
issues and discuss its with others 
using proper argumentation 

7(16%) 

ability to use scientific 
knowledge when solving 
everyday life issues 

(statement 3); 


Table 4 shows that for statements where the mean value was 4, 00 or higher, the most 
frequently mentioned level among all study groups was the third level of scientific literacy. 
This is the case even allowing for the fact that approximately 50% statements were 
determined to be at the third level (as shown in table 3). Only teacher and students studying 
overseas gave high mean scores to about 50% of the items indicating a high uniformity of 
responses to these statements. On the other hand, for statements with a mean value of 3.00 or 
lower, the most frequently mentioned level was the second level of scientific literacy and 
again this was so, allowing for the suggestion as indicated in table 3 that 36% statements were 
at the functional level. Only the scientist group came close to seeing about a third of the 
statements being of relatively low importance, this percentage dropping to as low as 5% in the 
case of teachers. 

Table 4. The division of statements between the levels of scientific literacy among 6 study groups, 
given as number of statements and a percentage of the total statements at that level 

Number (% of total) statements 



Public 

sector 

employers 

Private 

sector 

employers 

Scientists 

Teachers 

Students 

studying 

abroad 

Grade 12 
students 

Mean value over 4,00 

Functional 

2(2%) 

1(6%) 

2(12%) 

5(30%) 

8(48%) 

4(24%) 

Conceptual/procedural 

10(50%) 

6(30%) 

6(36%) 

16(80%) 

11(55%) 

5(25%) 

Multidimensional 

2(29%) 

2(29%) 

2(29%) 

3(43%) 

3(43%) 

0 

Total 

14(32%) 

9(20%) 

10(23%) 

24(55%) 

22(50%) 

9(20%) 

Mean value under 3.00 

Functional 

5(30%) 

7(42%) 

10(60%) 

2(12%) 

5(30%) 

5(30%) 

Conceptual/procedural 

1(5%) 

4(20%) 

5(25%) 

0 

4(20%) 

3(15%) 

Multidimensional 

1(14%) 

1(14%) 

1(14%) 

0 

2(29%) 

1(14%) 

Total 

7(16%) 

12(27%) 

16(36%) 

2(5%) 

11(25%) 

9(20%) 


Further analysis of the results showed that multi-dimensional level statements, in general, 
were highly supported although only one statement was given a mean of 4.00 or greater by all 
groups (number 14, see below). In addition, 3 statements (numbers 1, 3, 22) were given a 
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mean value of 4.00 or greater by 3 groups. It was noticeable that teachers gave all these 
statements a mean greater than 4.00. 

Statement 14. Willingness in and interest for lifelong learning 

Statement 1. Ability to judge the current situation and take actions accordingly 

Statement 3. Ability to use scientific knowledge when solving everyday life issues. 

Statement 22. Ability to distinguish between science and pseudoscience. 


One multidimensional statement (number 17) was not given a high mean by any group and, in 
fact, 4 groups (public/private and both student groups) gave this statement a mean of 3.00 or 
lower. 


Statement 17. Ability to use different strategies when solving socio-scientific problems. 


The most highlighted level (the most number of statements given a high response) in all 
groups, i.e. the third level of scientific literacy, indicates that stakeholders are realistic about 
their expectations and know what they are looking for in a future employee. Nevertheless, 
responses towards specific statements were mixed. Of the 20 statements determined to be at 
level 3, all gained a mean of 4.00 or greater from at least one group, whereas half the 
statement were given a mean of 3.00 or less by a least one group. Only two statements 
(numbers 18 and 44) were give a mean of 4.00 or greater by all six groups, although 
statements 40 and 42 was given a high mean by all groups except scientists, and for these 
statements the response by scientists contrasted with the strong agreement especially by 
students. In fact, it is somewhat surprising that scientists should give low importance to 
showing initiative. 


Statement 18. Ability to use creativity in professional work related activities. 
Statement 44. Ability to co-operate with people with different skills and knowledge. 
Statement 40. Correct language usage when compiling tests. 

Statement 42. Showing initiative. 


Greater diversity was shown by responses to about half the statements, deemed to be at level 
3, where only a mean of 4.00 or more was given by one group, (usually teachers), typically 
with about 2 groups giving a mean of 3.00 or lower (often students). Statement number 9 was 
the least supported, with 4 groups (private, scientists and both student groups) giving a mean 
of 3.00 or less. A possible interpretation here is that the statement is not sufficiently 
meaningful to the stakeholders. 


Statement 9. Ability to use scientific knowledge to predict possible changes in the environment. 


As expected, fewer statements deemed to be at the functional literacy level were given high 
mean values - a mean value of 4.00 or higher - by any group. However, one statement 
(number 30) was rated highly by all groups and another (number 28) rated as quite high by all 
groups. These are perhaps seen as attitudinal statements rather than as a skill. 


Statement 28. Ability to accept others opinions. 

Statement 30. Ability to seek help when solving different problems 


Contrasting this, however, only 4 functional level statements were not given a mean value of 
3.00 or less by at least one group. Statement (number 19) was rated low by all groups, with 
statement numbers 8 and 37 by all groups except teachers (number 8) and scientists (number 
37). The response by teachers is not unexpected, as formulas and laws are usual aspects in 
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the curriculum, but it is surprising that scientists give low importance to mathematical 
thinking in physic and chemistry. 

Statement 8. Ability to use formulas and laws taught in school. 

Statement 19. Gaining Technical drawing skills. 

Statement 37. Ability to solve physics and chemistry assignments using mathematical thinking. 


The emphasis on the functional level, at least by teachers and students, may be caused by the 
overall nature of the science education system where the national examination still 
emphasizes factual knowledge more than both practical skills and the ability to use 
knowledge in problem solving. For example: 


Statement 15 Ability to use modern technical equipment with manual. 

Statement 16. Ability to memorize and reproduce facts and formulas taught in school. 
Statement 21. Knowledge of symbolic language (writing formulas and chemical reactions). 
Statement 36. Strong factual knowledge in chemistry, physics and mathematics. 


Summary 

From a consideration of statements attracting a mean of 4.00 or greater in table 4, all groups 
supported statements at level 3 (conceptual scientific literacy) as the preferred learning, with 
teachers and students giving more emphasis to this level. It seems there is little difference 
between the views of Estonian students studying abroad and grade 12 students studying in 
Estonia; the difference is only strongly apparent in 3 statements (numbers 3, 22 and 39). 

The third level of scientific literacy is important for both employers and employees, because it 
enables the employees to be more independent and employers to get gain conceptually able 
employees, or as one participant wrote: 

“ ...we need someone who can think with his own head and act according to the situation. ” 

For the society it means that more people are capable of being interested in what is actually 
happening in their civic life and using proper argumentation and skills gained from science 
education to make one’s life better. 

The least important competencies and knowledge (lower part of table 4), in the opinion of 
participant, are mainly from the second level of scientific literacy. It suggests that most 
knowledge and skills provided in school is rather factual and not meeting employer’s 
expectations. Students, in particular, recognized this. Nevertheless, the relatively small 
number of statements with a mean under 3.00 (only scientists gave over one third of 
statements low importance) provides positive information on stakeholders’ expectations 
related to valuing a range of different skills and competencies. Statements at the 
multidimensional level rarely attracted a mean value less than 3.00 and statement 17 (with 
mean values less than 3.00 by 4 groups) was probably misunderstood. 

When working with the Delphi method, there are some limitations (see Table 1) which this 
study tried to avoid. For example, when introducing the method to participants some said that 
they would not have enough time to participate, which reduced the number of participants 
because of dropping out during the study. The involvement of the experts from the science 
education field helped to compose the participant sample and this lessened the possibility of 
including stakeholders’ who might drop out because they feared their expertise may not be as 
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expected. Also seeking experts’ opinion in modifying the initial question helped to minimize 
opportunities for misunderstanding. 

Conclusion 

The aim of this study was to find out whether data collected by the Delphi method can be 
interpreted in the light of a consideration of levels of scientific literacy. This study showed 
that, when precautionary steps in using the Delphi method are taken into account, learning 
expectations can be interpreted based on levels of scientific literacy. Although the levels were 
more specifically worded for the study, it was possible to organize the statements within the 
four levels of scientific literacy, as stated by Bybee (1997). 

When comparing stakeholders’ views regarding scientific literacy with those brought out in 
the literature (e.g. Fensham, 2007; Bybee, 1997, Holbrook and Rannikmae, 2009), it is clear 
that, in many cases, the stakeholders do emphasize those competencies and skills which are 
thought to be essential for a scientifically literate person. General, the outcomes from the 
small group of students studying abroad are comparable with the enlarged group of grade 12 
students, thus giving confidence in accepting the Delphi study outcomes These outcomes 
show that knowing facts and laws still has an impact on competencies expected, especially by 
teachers and students. This is reflected by the emphasis on the functional level of scientific 
literacy by science teachers and the students. 

It is important to continue the study with a greater number of stakeholders and find out which 
of the competencies and knowledge is essential for both future employees and citizens, if the 
aim of the society is a movement towards a knowledge based society (as stated in Lisbon 
Strategy, 2002) where scientifically literate persons are needed. 

The use of the Delphi approach, paying careful attention to its disadvantages, is shown to be a 
good way of soliciting the views of stakeholders. It would be advantageous, using this 
method, to extent the range of stakeholders to include policy-makers, such as politicians and 
those in educational leadership. 

This research is supported by ESF grant GL0L08219 and EDUKO grant LoTeGum. 
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Appendix 1. The mean values for the 44 statements with levels of scientific literacy (SL) among the 6 participating groups. 


1 st 


Statement 

Level 
of SL 

group 

Mean 

* 

SD 

2 nd 

group 

Mean 

1. Ability to judge the current situation and 
take actions according to it 

4* 

4,00 

1,03 

4,33 

2. Ability to examine/observe processes or 
phenomena in the nature 

3 

3,13 

1,38 

3,50 

3. Ability to use scientific knowledge when 
solving everyday life issues 

4* 

3,75 

0,75 

3,17 

4. Acknowledging science as a part of society 

3 

4,00 

0,52 

3,67 

5. Ability to speak in public about issues that 
demand scientific thinking 

4* 

4,25 

0,98 

3,83 

6. Ability to carry a logical conversation on 
important topics in contemporary science 

4* 

3,75 

0,84 

3,50 

7. Teamwork skills when solving problems 

3 

3,50 

0,75 

3,17 

8. Ability to use formulas and laws taught in 
school 

2 

2,38 

1,37 

2,33 

9. Ability to use scientific knowledge to 
predict possible changes in environment 

3 

3,25 

1,26 

3,00 

10. Ability to find information about science 
and technology from foreign sources 

2 

3,38 

1,17 

2,83 

11. Orienting in ICT equipment used in 
knowledge-based societies 

3 

3,00 

1,03 

2,67 

12. Ability to plan and conduct a research 
independently 

3 

3,75 

1,6 

2,83 

13. Ability to analyze problems with scientific 
content raising in society 

3 

4,00 

1,33 

2,83 

14. Willingness and interest for lifelong 
learning 

4* 

4,00 

0,75 

4,17 

15. Ability to use modern technical equipment 
with manuals 

2 

3,88 

1,38 

3,50 

16. Ability to memorize and reproduce facts 
and formulas taught in school 

2 

2,63 

0,84 

2,20 

17. Ability to use different strategies when 

4* 

2,63 

1,05 

2,50 


3 rd 4 ,h 5 th 6 th 


group group group group 


SD 

Mean 

SD 

Mean 

SD 

Mean 

SD 

Mean 

SD 

0,93 

3,60 

0,89 

4,50 

0,84 

3,67 

1,15 

3.94 

0,80 

1,36 

2,80 

1,1 

4,33 

0,82 

3,33 

1,15 

3,00 

0,84 

1,16 

4,00 

1 

4,67 

0,52 

4,33 

0,58 

3,13 

0,98 

0,76 

2,40 

1,52 

3,33 

0,52 

4,00 

1 

3,41 

0,95 

0,89 

3,00 

1 

3,83 

0,75 

3,00 

1 

3,47 

0,95 

0,89 

3,60 

0,89 

3,50 

0,84 

4,00 

1,73 

3,28 

1,02 

1,07 

3,00 

1,87 

3,83 

0,75 

4,00 

0 

3,84 

0,88 

0,52 

3,00 

0,71 

3,67 

1,21 

3,00 

1,73 

2,91 

1,17 

0,89 

2,60 

1,14 

4,33 

0,82 

2,67 

1,15 

2,88 

1,04 

1,19 

1,80 

0,84 

3,33 

0,82 

4,00 

1 

3,50 

1,02 

0,93 

3,25 

0,5 

4,00 

1,26 

2,67 

0,58 

3,06 

1,05 

0,89 

2,00 

1,41 

3,50 

0,55 

4,00 

1 

3,45 

1,21 

0,89 

3,60 

0,89 

3,83 

0,98 

3,00 

1 

2,88 

0,98 

0,52 

4,60 

0,55 

4,83 

0,41 

4,67 

0,58 

4,26 

0,89 

1,25 

2,80 

1,1 

4,67 

0,52 

4,00 

1 

4,09 

0,89 

1,06 

3,20 

0,84 

3,33 

1,03 

3,00 

1 

2,72 

0,77 

1,19 

3,20 

0,45 

3,50 

0,55 

2,67 

1,15 

2,72 

0,85 
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solving socio-scientific problems 

18. Ability to use creativity in professional 


work related activities 

3 

4,25 

0,82 

4,33 

0,89 

4,60 

0,55 

4,33 

0,52 

4,67 

0,58 

4,00 

0,88 

19. Technical drawing skills 

20. Critical evaluation of scientific texts 

2 

2,13 

0,75 

2,17 

1,36 

1,60 

0,55 

2,67 

0,52 

2,33 

0,58 

2,81 

1,18 

(distinguishing reliable information) 

21. Knowledge in symbolic language (writing 

3 

3,88 

0,89 

4,00 

1,36 

3,60 

0,55 

4,50 

0,55 

4,00 

1 

3,34 

1,10 

formulas and chemical reactions) 

22. Ability to distinguish between science and 

2 

2,13 

0,75 

1,83 

1,36 

2,60 

1,14 

3,17 

0,98 

3,00 

1,73 

3,03 

1,06 

pseudoscience 

4 * 

3,75 

1,22 

3,50 

1,16 

4,40 

0,89 

4,17 

0,75 

4,33 

1,15 

3,28 

1,17 

23. Basic knowledge in technology 

24. Using correct scientific terminology when 

2 

3,38 

1,37 

3,67 

0,92 

3,20 

0,45 

3,83 

0,98 

3,00 

1 

3,66 

1,12 

expressing ones opinion 

25. Ability to find scientific information 

3 

3,63 

0,82 

3,67 

1,06 

3,60 

0,55 

4,33 

0,52 

3,00 

0 

3,06 

1,13 

without assistance 

26. Ability to make scientifically reasoned 

2 

4,38 

0,55 

3,50 

0,52 

3,80 

0,84 

4,67 

0,52 

4,00 

1 

3,72 

0,89 

decisions when solving everyday life issues 

27. Ability to see different knowledge gained 

3 

2,75 

1,03 

3,33 

1,28 

3,00 

0,71 

4,00 

0,63 

3,67 

0,58 

3,53 

0,80 

in science classes as a whole 

3 

4,13 

0,89 

4,00 

0,99 

3,20 

0,84 

4,67 

0,52 

4,00 

1 

3,53 

1,02 

28. Ability to accept others opinion 

29. Taking economic aspects into account 

2 

3,63 

0,75 

3,83 

1,19 

4,40 

0,55 

4,33 

0,82 

4,67 

0,58 

4,37 

0,76 

when solving scientific problems 

30. Ability to seek help when solving different 

3 

3,13 

0,41 

3,83 

0,99 

3,00 

0 

4,00 

0,63 

3,67 

0,58 

3,50 

0,98 

problems 

31. To reason one’s opinion scientifically 

2 

4,38 

0,41 

4,83 

0,74 

4,25 

0,5 

5,00 

0 

4,33 

1,15 

4,16 

0,99 

correct 

3 

3,63 

0,75 

3,83 

0,92 

3,50 

0,58 

3,83 

0,41 

4,00 

1 

4,00 

0,92 

32. Ability to formulate research questions 

33. Using laws and normative when making 

3 

4,13 

0,75 

3,17 

1,13 

3,75 

0,96 

3,33 

0,82 

3,67 

1,15 

3,69 

1,00 

environmental decisions 

34. Planning and conducting research with 

2 

3,25 

0,98 

3,17 

1,16 

2,80 

0,84 

4,50 

0,55 

3,33 

0,58 

3,38 

0,98 

assistance or guidance from others 

35. Ability to distinguish between scientific 

2 

3,75 

0,82 

2,67 

1,04 

2,60 

0,89 

3,83 

1,17 

4,00 

1 

3,66 

1,10 

facts and subjective opinion 

36. Strong factual knowledge in chemistry. 

3 

4,13 

1,03 

3,33 

0,83 

4,00 

1 

4,17 

0,75 

4,00 

1 

3,69 

1,06 

physics and math 

37. Ability to solve physics and chemistry 

2 

3,38 

1,21 

3,67 

1,19 

2,80 

0,89 

3,50 

0,55 

3,33 

1,53 

2,78 

1,21 

assignments using mathematical thinking 

2 

2,38 

1,03 

2,33 

0,74 

3,40 

1,14 

2,67 

1,21 

2,67 

1,15 

2,41 

1,04 
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38. Ability to understand and explain 


phenomena and processes in the nature 

39. Ability to read and understand specialized 

3 

4,00 

0,98 

3,17 

1,07 

4,40 

0,55 

4,33 

0,82 

3,67 

0,58 

3,13 

0,98 

foreign texts 

40. Correct language usage when compiling 

2 

3,50 

0,82 

3,67 

1,2 

2,80 

0,84 

3,67 

1,37 

4,33 

0,58 

3,44 

1,01 

texts 

41. Ability to recognize science content in 

3 

4,38 

1,17 

4,17 

1,06 

3,60 

0,55 

4,33 

0,52 

4,33 

1,15 

4,00 

0,92 

everyday life issues 

3 

3,50 

1,03 

3,33 

1,07 

4,00 

0,71 

4,33 

0,52 

3,00 

1 

3,34 

1,15 

42. Initiative 

3 

4,13 

0,52 

4,67 

1,36 

3,80 

0,84 

4,17 

0,41 

5,00 

0 

4,19 

0,93 

43. Mathematical thinking 

44. Ability to co-operate with people with 

2 

3,38 

0,55 

3,50 

0,92 

3,20 

0,45 

3,33 

0,52 

3,67 

0,58 

3,23 

1,12 

different skills and knowledge 

3 

4,25 

0,41 

4,83 

1,16 

4,80 

0,45 

4,33 

1,63 

4,67 

0,58 

4,39 

0,72 


* The numbers indicate the participating groups (1-pivate sector; 2-public sector; 3-scientists; 4- teachers; 5- Estonian students studying aboard; 6- 12 grade students). 
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